Ambient Shadow Detection and Removal via Flash/Noflash Pairs

Flash/noflash pairs have been studied as a simple mechanism for supplying less-noisy information for dark areas of an ambient-light image. This research direction also includes image pairs or sequences of nighttime or obscured imagery such as surveillance video. Ambient imagery has some advantages over flash images, in general, in that a flash can create harsh or unpleasing effects, and can also generate additional shadowing. The extra shadows from the flash are in fact quite amenable to detection. However, filling in information in image areas occupied by shadows in the ambient image has not been effectively considered. Here we consider the problem of detecting shadows in the image taken under ambient light, given extra information from a flash image registered with the first. Clearly, the pure-flash image is the difference between the two — and has no ambient-shadow component. But the question of how best to detect the ambient shadow remains. We argue that first going to a “spectrally sharpened” color space, and then focusing on the difference in a log domain of the flash image and the ambient image, gives a very simple feature space consisting of two components — one in an illuminant-change 3-vector direction, and one along the gray axis. This space provides excellent separation of the shadow and nonshadow areas. Regressing pixel data from the flash image to the nonflash one adjusts the scale properly, and then inserting edges from the flash image inside the ambient-shadow region into the ambient image edge map and inverting Poisson’s equation fills in the shadow. In this way, we arrive at an image with the advantages of the ambient-only image — warmth, no flash effects such as disturbing illumination dropoff with distance and pixel saturation etc. — but no shadows.